1. Field of the Invention
The present invention relates to an image coding apparatus which is used in image communications equipment such as, visual telephone and teleconferencing equipment and image storage/recording equipment such as a digital VTR.
The invention also relates to a motion prediction circuit which performs motion detection and motion prediction on image information as well as to image coding and decoding apparatuses to which the above motion prediction circuit is applied.
2. Description of the Related Art
Conventionally, various image coding techniques have been developed to improve the efficiency of transmission and recording of digital images.
FIG. 9 is a block diagram showing a prior art image coding apparatus that is shown in Recommendation H.261 of ITU-T (International Telegraph and Telephone Consultative Committee). In this apparatus, first an input digital image signal 101 to be coded is input to a differentiator 21. The differentiator 21 takes a difference between the input digital image signal 101 and a prediction signal 136 (described later), and outputs a resulting difference signal as a prediction error signal 131. A coding section 22 encodes the prediction error signal 131 and outputs resulting code data 132. The above Recommendation employs, as a coding method in the coding section 22, a technique of converting the prediction error signal 131 from the spatial domain to the frequency domain by using DCT (discrete cosine transform) that is a kind of quadrature transform, and linearly quantizing resulting transform coefficients.
The code data 132 as output from the coding section 22 in branched into two parts, one of which is sent to a reception-side decoding apparatus (not shown). The other part is input to a decoding section 23 of the image coding apparatus under discussion. The decoding section 23 performs an inverse operation to the operation of the coding section 22, that is, it produces a decoded prediction error signal 133 based on the code data 132. An adder 24 adds the decoded prediction error signal 133 to the prediction signal 136, to thereby produce a decoded image signal 134. One frame of the image signal 134 is stored in a memory 25 such as a frame memory. The memory 25 outputs the stored decoded image signal of one frame after delaying it by one frame, as a decoded image signal 135 of a preceding frame. A predictor 26 produces the prediction signal 136 and a motion vector 137 by performing motion-compensating prediction based on the input digital signal 101 to be coded and the 1-frame preceding decoded image signal 135. The motion vector 137 is sent to the reception-side decoding apparatus (not shown) and the prediction signal 136 is supplied to the differentiator 21 and the adder 24.
In the conventional image coding apparatus having the above configuration, the coding section 22 encodes an image of one frame substantially uniformly irrespective of the content of a subject image. Further, the image 25 operates such that an image of only one frame is stored therein and rewritten every frame. Therefore, the efficiency of coding cannot be improved in the conventional image coding apparatus.
A prior art motion prediction circuit is disclosed in Japanese Unexamined Patent Publication No. Hei. 4-347987, which is shown in FIG. 19. In FIG. 19, reference numeral 611 denotes a motion vector detecting circuit; 612, a segmenting circuit; and 613, a motion parameter detecting circuit. Further, reference numeral 601 denotes an input image signal; 602, interframe motion vectors; 603, segmentation information; and 604, a motion parameter that is detected on a-segment-by-segment basis.
The above circuit operates in the following manner. An input image signal 601 is supplied to both the motion vector detecting circuit 611 and the segmenting circuit 612. The motion vector detecting circuit 611 detects a motion vector 602 on a pixel or small-block basis and supplies the detected motion vectors 602 to the motion parameter detecting circuit 613. On the other hand, the segmenting circuit 612 divides the input image signal 601 into a plurality of segments having different motion by dividing or combining blocks by referring to the motion vectors 602, and outputs segmentation information 603 indicating a manner of segmentation. The motion parameter detecting circuit 613 selects motion vectors belonging to a segment from the motion vectors 102, and calculates first-order conversion coefficients which describe motion of the segment based on the selected motion vectors. By performing this operation for all the segments, the motion parameter detecting circuit 613 outputs detected motion parameters 604 as motion information of the input image.
FIG. 20 shows details of the motion parameter detecting circuit 613. First, coordinates of N measurement points belonging to a target segment of motion parameter detection are selected. The N sets of measured coordinates are supplied to a center-of-gravity detecting circuit 621, which calculates coordinates of the center of gravity of the N measurement points.
The center-of-gravity detecting circuit 621 then converts the coordinates of the N measurement points into coordinates relative to the calculated center of gravity, and supplies the resulting coordinates to average detecting circuits 622-624. Further, x-components Vx(X, Y) of the N measured motion vectors are input to the average detecting circuit 622, and y-components Vy(X, Y) are input to the average detecting circuit 624. The average detecting circuits 622-624 calculate various averages and supply those averages to first-order conversion coefficients detecting circuits 625 and 626. The first-order conversion coefficients detecting circuit 625 calculates first-order conversion coefficients a, b, and e by a method of least squares based on the received averages. Similarly, the first-order conversion coefficients detecting circuit 626 calculates first-order conversion coefficients c, d, and f. These coefficients are used as motion parameters of the segment concerned. Thus, the motion parameters of each segment of the input image are obtained.
Another conventional motion prediction circuit is disclosed in Japanese Unexamined Patent Publication No. Hei. 5-328334. This publication discloses a block coding scheme in which coding is performed in units of enlarged blocks each obtained by combining similar adjacent blocks.
In the first conventional example described above, a method of least squares is used in converting the motion vectors 602 into the motion parameters 604, i.e., first-order conversion coefficients. This is based on the assumption that the motion of a segment is sufficiently small. Therefore, detection errors become large when the motion is large.
Further, coordinates of N measured points belonging to the target segment are needed to produce the motion parameters 604. In the case of real-time communication of moving images, this requires that motion vectors of each segment of an input image be measured automatically at high speed with high accuracy, which is not realistic.
Still further, there is no criterion to judge whether optimum segments have been obtained for the motion vectors detecting operation. Therefore, the prediction performance strongly depends on the accuracy of segmentation.
On the other hand, in the second conventional example described above, coding is performed on a block-by-block basis in which a plurality of blocks are combined into a larger block in a fixed manner. Therefore, the prediction error of an image cannot be made smaller than a block.
The present invention has been made to solve the above problems in the art and, therefore, has an object of providing an image coding apparatus which can improve the efficiency of coding by coding an image in accordance with its content and rewriting the contents of memories in accordance with the content of an image.
Another object of the invention is to provide a motion prediction circuit which can correctly perform motion prediction with a prediction error smaller than a block, and which can always perform correct motion prediction without the needs of high-speed calculation and obtaining measured values.
According to a first aspect of the invention, there is provided an image coding apparatus comprising:
a segmenting section for dividing an input image into a plurality of segments, and producing an image signal such that the respective segments are distinguished from each other;
a hierarchizing section for determining classes of the respective segments according to a predetermined criterion, and producing a class identification signal indicating the classes of the respective segments; and
a coding section for coding the image signal into code data while changing a value of a coding control parameter for each of the segments in accordance with the class identification signal.
Further, there is provided an image coding apparatus comprising:
a segmenting section for dividing an input image into a plurality of segments, and producing an image signal such that the respective segments are distinguished from each other;
a hierarchizing section for determining classes of the respective segments according to a predetermined criterion, and producing a class identification signal indicating the classes of the respective segments;
a plurality of coding sections for coding the image signal into code data according to different coding schemes; and
a switch section for selecting one of the plurality of coding sections for each of the segments in accordance with the class identification signal.
Further, there is provided an image coding apparatus comprising:
a segmenting section for dividing an input image into a plurality of segments, and producing an image signal such that the respective segments are distinguished from each other;
a hierarchizing section for determining classes of the respective segments according to a predetermined criterion, and producing a class identification signal indicating the classes of the respective segments;
a plurality of memories for storing a past image signal;
a prediction section for reading, on a segment-by-segment basis, a prediction source image signal that is similar to the image signal produced by the segmenting section from the plurality of memories, and for producing a prediction signal by predicting a current-frame image signal on a segment-by-segment basis based on the prediction source image signal;
a prediction error extracting section for extracting a prediction error signal on a segment-by-segment basis based on the prediction signal and the actual image signal of a current frame that is produced by the segmenting section;
a coding section for coding the prediction error signal into code data;
a decoding section for decoding the coded data into a decoded prediction error signal;
a signal combining section for combining the decoded prediction error signal and the prediction signal into a decoded image signal; and
a memory control section for storing portions of the decoded image signal corresponding to the respective segments into the plurality of memories while switching among the plurality of memories in accordance with the class identification signal, to thereby update storage contents of the plurality of memories.
According to a second aspect of the invention, there is provided a segmentation-type motion prediction circuit comprising:
first segmenting means for segmenting, when necessary, each of blocks of an input image signal into a plurality of segments by extracting a feature of the input image signal on a block-by-block basis;
motion vector detecting means for detecting motion vectors for the respective segments;
segment boundary line approximating means for determining a pattern and a level of each of the blocks from a boundary line between the segments belonging to the block;
motion vector re-search means for adjusting motion vectors of the respective segments based on the pattern and the level, and for producing segment boundary line information, final motion vectors, and prediction image information.
There is provided an image coding apparatus comprising:
a segmentation-type motion prediction circuit comprising:
segmenting means for segmenting each of blocks of an input image signal into a plurality of segments by extracting a feature of the input image signal;
motion vector detecting means for detecting motion vectors for the respective segments;
segment boundary line approximating means for determining a pattern and a level of each of the blocks from a boundary line between the segments belonging to the block; and
motion vector re-search means for determining motion vectors of the respective segments based on the pattern and the level, producing prediction image information, and transmitting the final motion vectors and segment boundary line information;
quantizing means for quantizing transform coefficients that is obtained by transform-coding a difference between the input image signal and the prediction image information;
dequantizing means for dequantizing the quantized transform coefficients;
a frame memory for storing, as a decoded image signal of a current frame, a sum of a decoded image signal of a preceding frame and a prediction error image signal that is obtained by inverse-transform-decoding the dequantized transform coefficients; and
variable-length coding means for coding the quantized transform coefficients and transmitting resulting coded image data.
There is provided a segmentation-type motion-predictive image decoding apparatus comprising:
dequantizing means for dequantizing decoded transform coefficients of transmitted or reproduced coded image data;
inverse transform decoding means for inverse-transform-decoding the dequantized transform coefficients into a prediction error image signal;
segment boundary line decoding means for producing a boundary line of each block from a transmitted or reproduced segment boundary line information; and
prediction image generating means for generating a prediction image signal from the boundary line, transmitted or reproduced motion vectors of respective segments, and a decoded image signal of a preceding frame; and
means for producing a decoded image signal from the prediction error image signal and the prediction image signal.